The invention relates to data processing systems and especially to using person-based metadata in data processing.
The amount of personal information processed daily has continued to increase after the introduction of digital data processing. For instance, the facts that digital cameras have become more common and their quality improved have changed the way people take photographs. They take large numbers of photographs, even thousands in a year, and want to look at them, process them and share them with others. It is clear that powerful tools are needed to manage the large daily amounts of information.
However, it has also been noticed that managing masses of photos by using automatic functions only is not sufficient. A computer can generate technical data related to the size, structure and transmission format of the information being processed, but a computer cannot efficiently process information on the basis of the significance attached to its content by the person using it.
To be able to fetch and browse digital data elements automatically, metadata need to be added to them, i.e. information about the data and especially about its content. There are two approaches to providing contentual metadata: the metadata can be automatically generated from the information or the user processing the information can be asked to attach, i.e. annotate, the metadata manually.
For instance, the metadata that can automatically be separated from a photograph by using current methods is typically not very abstract and of low quality, such as technically defined information on the colours, forms, and the like in the photograph. The metadata that is relevant for the user is, however, semantic by nature and more abstract. This means that users want information on for instance who the people are in the photo, or what they do. This difference between the automatically available metadata and the metadata required by the users is called a semantic gap.
Due to the above-mentioned problem, it is typically necessary to settle for the provision of manual (or semi-automatic/assisted) metadata. According to studies, users are very reluctant to annotate manually, because it is a time-consuming process that is susceptible to errors and considered boring. This problem is emphasized for instance when using a phone equipped with a camera, because the small display and limited input means of the phone make the annotation of photos difficult and slow it down. On the other hand, people feel that the annotated content is more useful and are later on pleased with having annotated their photos.
Publication “Requirements for Photoware” by David Frohlich, Allan Kuchinsky, Celine Pering, Abbe Don & Steven Ariss, Proceedings of the 2002 ACM conference on Computer supported cooperative work, examines the weaknesses and strengths of the present image material sharing methods and studies the needs of the users in processing image material through prioritization. The study recognizes the users'reluctance for manual annotation and recommends as a solution the use of collections and audio annotations attached thereto.
Publication “Ontology-based Photo Annotation” by A. Th. (Guus) Schreiber, Barbara Dubbeldam, Jan Wielemaker, and Bob Wielinga, IEEE Intelligent Systems, May/Jun. 2001, describes a photo annotation tool for processing photos in RDF (Resource Description Framework) file format. The description of the photo content is done using a model based on structural annotation by utilizing an ontology created for this purpose.
Publication “Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos” by Den Scneiderman, Hyunmo Kang, IEEE International Conference on Information Visualisation (IV2000), describes a user interface in which an image or series of images can be annotated by attaching to them free text fields defined by the user and other information received through the user interface, such as dates and geographical information.
Contentual metadata created by different persons is generally not commensurable, because a common metadata ontology is not available. For instance, when processing image data, the same person can be referred to in different names and, on the other hand, one name can refer to several persons. Thus, it is not generally possible to utilize metadata created by other users in the prior art solutions.
For instance, the most typical subjects of personal photography are the people closest to the photographer and especially events where these people are present. Therefore, the most significant need for contentual metadata is also related to people and events. People are most interested in photos in which they are present or which relate to events they were participated in. Thus, when there are a lot of photos—as typically happens in the case of a digital camera and a phone equipped with a camera—the content of an individual photo is not primarily important for usability, but instead the possibility to easily fetch the photos related to a certain event or person. In existing solutions, the persons in the annotations need, however, to be annotated separately every time, or if a full-scale user interface is available, the annotation can be added from a drag-and-drop menu created in advance by the user. This is an arduous solution for the user, and it is, in practice, not feasible in the more limited user interfaces.
Photographs are often very personal and confidential by nature. Typically, there is a clear target group for the photos, which varies strongly between different photos of the same photographer. Often this target group is made up of the same people that were present in the event, for instance party, related to the photos. By modern methods, the sharing of even digital photos is difficult and is based primarily on sending separate image files by e-mail or storing them at a certain network address. In such cases, access to the image material is managed by other definitions than those related to the content of the photo.
In summary, a problem with the present data processing systems is that the person-based metadata that controls the later use of the information is needed in a format that is as widely recognized as possible, but so as to generate contentual metadata, the attachment of the metadata should be as user-friendly as possible. Attaching metadata should preferably be implemented over the simplest possible user interface, such as the user interface of a mobile terminal.